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Reconsideration of the application is respectfully requested in view of the foregoing 
amendments and following remarks. 

Patentability over Moriya et al. 

Claims 1-37 have been rejected under 35 U.S.C. 102(e) as being anticipated by Moriya et al., 
U.S. Pat. No. 6,046,745. The rejections are respectfully traversed. 

Claims 1-8 

Claim 1 is directed to a method of generating a three-dimensional scene from a sequence of 

two-dimensional images. As amended, claim 1 recites, 

A method of recovering a three-dimensional scene from two-dimensional images, the 
method comprising: 

providing a sequence of images; 

dividing the sequence of images into segments, the segments comprising groups of 

frames ; 

performing three-dimensional reconstruction for each segment individually; and 
combining the three-dimensional reconstructed segments together to recover a three- 
dimensional scene for the sequence of images. 

Claim 1 's method includes the element, "dividing the sequence of images into segments, the 
segments comprising groups of frames." For example, in one embodiment described in the 
specification, a long sequence of images is sub-divided into segments containing a subset of the 
entire sequence of images in order to reduce the complexity of processing the entire sequence. 

Moriya et al. fails to teach or suggest all elements of claim 1 . Specifically, Moriya et al. fails 
to teach or suggest "dividing the sequence into segments" as recited in claim 1. Moriya et al. also 
fails to teach that segments comprise "groups of frames." In the action, the office suggests that 
Moriya et al. teaches, "dividing the sequence of images into segments" and directs the applicants to 
element 4038 in Fig. 40 in Moriya et al. The applicants respectfully disagree. 

Element 4038 of Fig. 40 in Moriya et al. refers to a "basic figure constituting a three- 
dimensional model concerning sub-figures such as points, lines, planes, type of a cube, a 
classification of a linear line and a curved line or the like." See Moriya et al., Col. 29, Lines 28-31. 
Thus, Moriya et al. does not teach "dividing the sequence of images into segments." Instead, Moriya 
et al. refers to providing "a configuration of component elements" of a single image or frame. See 
Moriya et al. Col. 25, lines 27-38. 
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Moriya et al. divides an image, whereas the claimed invention divides a sequence of images. 
Dividing a single image into component elements for generating a three-dimensional model, as 
taught by Moriya et al., is not the same as dividing a long sequence of multiple images into shorter 
segments to simplify the processing necessary to reconstruct a three-dimensional scene from a 
sequence of two-dimensional images. For example, each two-dimensional image in a sequence of 
images, as claimed by applicants, may be related to a different camera pose such that a collection of 
such images could be combined to generate a complete three-dimensional image. The examiner has 
failed to point out where Moriya et al. teaches a sequence of images. 

Because Moriya et al. fails to teach or suggest dividing a sequence of images into smaller 
segments, said segments comprising groups of frames, claim 1 and its dependent claims 2-8 should 
clearly be allowable over the cited reference. Such action is respectfully requested. 

Claims 9-22 

Claim 9 is also directed to a method of recovering a three-dimensional scene from two- 
dimensional images. Claim 9 also contains the following elements: 

dividing the sequence of frames into segments, wherein a segment 
includes a plurality of frames ; 

for each segment, encoding the frames in the segment into at least two 
virtual frames that include a three-dimensional structure for the segment and an 
uncertainty associated with the segment. 

Thus, at least for the reasons stated for claim 1, Moriya et al. does not teach methods of 
processing a sequence of two-dimensional images by dividing the sequence into segments prior to 
reconstructing a three-dimensional scene from the segments. 

Furthermore, claim 9's method includes the element, "encoding the frames in the segment 
into at least two virtual frames." For example, in one embodiment described in the specification, at 
least two frames from a segment are chosen that are a predetermined number of frames apart. See 
element 140 in Fig. 8. In the action, the office suggests that Moriya et al. teaches, "means for 
extracting virtual key frames from each partial model" and directs applicants to element 4036 in Fig. 
40 in Moriya et al. The applicants respectfully disagree. 

Element 4036 in Fig. 40 in Moriya et al. refers to "a three-dimensional extracting algorithm, 
i.e., making reference to: two-dimensional coordinate values seen on the image; camera parameters; 
various knowledge information of an object seen in an image which is vertical to the ground surface 
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or which is a rectangular parallelopiped or the like; and features of a basic figure constituting a 
three-dimensional model. . . ." See Moriya et al., Col 29, Lines 23-29. Thus, Moriya et al. does not 
teach a "means for extracting virtual key frames." Instead, Moriya et al. refers to providing a means 
for "designation of three-dimensional CG model data" of a single image or frame. See Moriya et al., 
Col. 29, Lines 21-22. Extracting three-dimensional CG model data from a single image or frame is 
not the same as selecting at least two frames which are representative of a larger segment of frames. 

Because Moriya et al. fails to teach or suggest dividing a sequence of images into smaller 
segments, and because Moriya et al. fails to teach or suggest extracting virtual frames, claim 9 and 
its dependent claims 10-22 should be allowable. Such action is respectfully requested. 

Claims 23-30 

With the above requested amendment, claim 23, as amended, recites the following element: 
"segmenting the sequence of two-dimensional frames into segments, the segments comprising 
groups of frames." Thus, at least for the reasons stated for claim 1, Moriya et al. does not teach or 
suggest dividing a sequence of two-dimensional images into subsets prior to reconstructing a three- 
dimensional scene from the subsets. Claim 23 and its dependent claims 24-30 should be allowable. 
Such action is respectfully requested. 

Claims 31-35 

As amended, claim 31 recites the element of "dividing a long sequence of frames into 
segments, the segments comprising groups of frames." Thus, at least for the reasons stated for claim 
1, claim 31 should be allowable. Furthermore, claims 32-35 depend on claim 31, thus claims 32-35 
should also be allowable. Such action is respectfully requested. 

Claims 36-37 

As amended, claim 36 is directed to a computer-readable medium having instructions for 

performing a method for recovering a three-dimensional scene from a sequence of two-dimensional 

frames by segmenting the frames. Claim 37, as amended, is directed to an apparatus for carrying out 

such a method. As amended, claim 36 recites, 

A computer-readable medium having computer-executable instructions for 
performing a method comprising: 

providing a sequence of two-dimensional frames; 
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dividing the sequence into segments, the segments comprising groups of frames: 
calculating a partial model for each segment that includes three-dimensional 

coordinates and camera pose for features within the frames; 

extracting virtual key frames from each partial model , the virtual key frames having 

three-dimensional coordinates for the frames and an uncertainty associated with the frames; 

and 

bundle adjusting the virtual key frames to obtain a complete three-dimensional 
reconstruction of the two-dimensional frames. 

Thus, at least for the reasons stated for claim 1, Moriya et al. does not teach methods of 
processing a sequence of two-dimensional images by dividing the sequence into segments prior to 
reconstructing a three-dimensional scene from the segments. 

Additionally, at least for the reasons stated for claim 9, Moriya et al. does not teach methods 
of extracting virtual key frames. 

Additionally, claim 36's method includes the element, "bundle adjusting the virtual key 
frames to obtain a complete three-dimensional reconstruction of the two-dimensional frames." For 
example, in one embodiment described in the specification, "Bundle adjustment is a non-linear 
minimization process applied to the segments to essentially perform an averaging of the segments 
together." Application, p. 7. 

Moriya et al. fails to teach or suggest "bundle adjusting the virtual key frames" as recited in 
claim 36. In the action, the office suggests that Moriya et al. teaches "bundle adjusting the virtual 
key frames" and directs the applicants to element 4040 in Fig. 40 in Moriya et al. The applicants 
respectfully disagree. 

Element 4040 of Fig. 40 in Moriya et al. refers to "CG model making software." See Moriya 
et al., Col. 29, Line 37. This phrase is so broad as to be incapable of specifically teaching a means 
for bundle adjusting. Furthermore, as explained above, Moriya et al. does not teach extraction of 
virtual key frames; therefore, it cannot teach bundle adjustment of the virtual key frames. 

Because Moriya et al. fails to teach or suggest dividing a sequence of images into smaller 
segments; because Moriya et al. fails to teach or suggest extracting virtual key frames; and because 
Moriya et al. fails to teach or suggest bundle adjusting the virtual key frames, claims 36 and 37 
should clearly be allowable over the cited reference. Such action is respectfully requested. 
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The claims in their present form should now be allowable. Such action is respectfully 
requested. 
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